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Abstract 

Knowledge  Management  for  Distributed-Tracking 
(KMDT)  is  an  ongoing  research  and  development 
project  to  to  improve  military-information  functions 
in  the  battle  space,  such  as  command,  control,  and 
decision  support.  It  features  a  scenario  that  shows 
how  knowledge-management  technologies,  such  as 
ontologies  and  intelligent  agents  can  improve  battle- 
space  awareness  and  the  decision-making  process  in 
command  centers  with  respect  to  .distributed  tracking 
and  threat  identification  of  platforms.  Cross  lines  of 
bearings  using  heterogeneous  sensor  data  and  other 
information  from  multiple  platforms  in  the  battle 
space  can  reduce  the  uncertainty  in  platform  detec¬ 
tion,  localization,  classification  and  identification. 
The  paper  describes  metrics  for  ontology  develop¬ 
ment  and  agent  performance. 

1.  Introduction 


formation  flows  in  the  battle  space  and  the  effect  of 
the  information  on  target  detection,  tracking  and  the 
ability  of  sensors  to  align  to  a  common  frame  of  ref¬ 
erence  in  time  and  in  space.  Not  only  can  multiple 
homogeneous  sensors  track  individual  platforms,  but 
also  multiple  sensor  types  can  participate  in  a  level- 
one  data  fusion  task  [10]  (e.g.  detection,  localization, 
classification,  and  identification)  coordinated  by  in¬ 
telligent  agents,  thus  reducing  uncertainty  in  com¬ 
mand  and  intelligence  centers.  According  to  the  pro¬ 
gram  plan,  agents  retrieve  data  needed  for  distributed 
heterogeneous  level-one  data  fusion  using  Lines  Of 
Bearing  (LOBs).  With  agent-based  medics,  the  effec¬ 
tiveness  of  agents  can  be  assessed  as  they  perform 
tasks  in  the  simulated  environment.  Another  key 
component  is  the  integrated  sensor  ontology  and  the 
metrics  for  its  development. 

2.  Motivation  for  KMDT 


The  goal  of  KMDT  is  to  explore  methods  to  im¬ 
plement  FORCE-net,  which  is  the  U.S.  Navy’s  opera¬ 
tional  construct  and  architectural  framework  for  naval 
warfare  in  the  information  age  [7].  The  goal  of  FOR- 
CEnet  is  to  integrate  warriors,  sensors,  command  and 
control,  platforms,  and  weapons  into  a  networked, 
distributed  combat  force  [7].  KMDT  assembles  tech¬ 
nologies  to  assess  the  information  content  exchanged 
in  the  battle  space  and  to  develop  enhanced  aware¬ 
ness.  This  will  enable  analysts,  operators,  and  warri¬ 
ors  alike  to  reduce  uncertainty  in  command  and  con¬ 
trol  by  better  organizing  and  using  the  data  collected 
from  existing  sensors. 

New  approaches  to  tracking,  command  and  con¬ 
trol  are  explored  using  knowledge-management  tech¬ 
nologies  such  as  sensor  ontologies  [4],  intelligent 
agents  [3],  ontology-development  metrics,  and  agent- 
based  metrics.  During  their  task  execution,  intelligent 
agents  can  access  sensor  ontology  to  obtain  informa¬ 
tion  relevant  to  current  sensor-data  requirements. 
Analysis  and  Monte  Carlo  simulation  can  assess  in¬ 


Fig,  1.  Platform  detection  geometry  showing 
lines  of  bearing  from  ships  A  and  B  detecting 
an  unknown  contact  with  heterogeneous 
sensor  types  1  and  2. 
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Sensors  deployed  on  a  single  platform,  such  as  a 
ship,  can  provide  LOB  information  on  unknown  con¬ 
tacts  and  potential  targets  in  their  vicinity  (Fig.  1). 
Cross  LOB  targeting  (i.e.  using  data  from  two  ships) 
either  is  not  done  or  it  is  limited  to  homogeneous 
sensor  systems  (e.g.  all  acoustic  sensors).  Thus,  in¬ 
formation  about  multiple  LOBs  that  could  localize 
the  position  of  a  target  often  does  not  reach  a  com¬ 
mand  center  in  time  to  support  the  decision  process. 
Sometimes  operators  do  not  know  what  to  do  with 
new  data  that  are  not  correlated  with  existing  data. 
Such  data  fail  to  reach  the  threshold  of  information  to 
support  decision  confidence. 

Commanders  and  sensor  operators  often  are  over¬ 
loaded  with  tasks  and  uncorrelated  information.  Con¬ 
versely,  they  sometimes  have  difficulty  in  obtaining 
the  correct  information  they  need  to  make  timely  de¬ 
cisions,  so  decisions  are  made  using  uncertain  infor¬ 
mation.  Local  data  are  lost  because  they  cannot  be 
correlated  with  data  from  remote  sensors  and  obser¬ 
vations  in  a  timely  manner.  Data  from  remote  sensors 
are  either  not  transmitted  efficiently  or  no  payoff  is 
perceived  for  their  propagation.  To  fespond  rapidly, 
the  commander  may  need  the  data  that  neither  are 
available  locally  nor  transmitted  from  remote  sensors. 

3.  Next-Generation  Command  &  Control 

This  section  describes  an  example  scenario  to 
show  how  the  capabilities  under  development  in 
KMDT  will  be  used  in  future  command-and-control 
operations.  A  commander  in  the  Combat  Information 
Center  (CIC)  on  board  northbound  ship  A  (Fig.  1) 
receives  a  sensor  report  of  a  contact  detected  at  a 
bearing  of  045  degrees.  The  unknown  contact  cannot 
be  classified  or  localized  with  only  the  information  in 
the  report.  An  operator  supporting  the  commander 
tasks  an  intelligent  agent  to  search  the  network  for 
friendly  platforms  in  the  battle  space  that  also  have 
detected  the  unknown  contact.  (See,  for  example  [8]). 

A  calculation  is  performed  in  the  agent- 
deployment  software  to  identify  the  search  spaces 
based  on  the  geometry  of  ship  A  and  the  contact.  The 
agent  finds  the  sensor-ontology  web  site  to  correlate 
the  known  capabilities  of  ship  B  with  the  kinds  of 
information  that  could  be  combined  with  the  sensor 
of  Ship  A  to  classify  the  unknown  contact.  The  agent 
searches  the  web  portal  for  Ship  B  to  find  LOB, 
acoustic  data,  and  the  date-time  group  of  the  sensor 
measurements  on  Ship  B  with  which  the  contact  was 
detected.  The  agent  discovers  that  Ship  B  has  de¬ 
tected  the  contact. 

The  agent  issues  an  alert  to  the  operator  on  ship  A, 
that  a  report  from  Ship  B  is  available.  The  informa¬ 


tion  retrieved  about  the  unknown  contact  enables  the 
operator  on  Ship  A  to  observe  the  LOBs  and  the  re¬ 
lated  data.  The  operator  fuses  this  information  with 
the  original  sensor  data  from  ship  A  and  recommends 
a  classification  (hostile,  friendly  or  neutral)  of  the 
unknown  contact  to  the  commander. 

4.  Background  and  Method 

KMDT  is  focused  on  knowledge-management 
technologies  such  as  sensor  ontologies,  and  intelli¬ 
gent  agents  to  upgrade  command  and  intelligence 
centers  to  provide  the  capabilities  described  in  the 
above  scenario.  The  modeling-and-simulation  effort 
is  conducted  to  demonstrate  the  feasibility  of  the 
technology.  During  the  simulation,  agents  access  web 
pages  that  represent  information  flowing  to  and  from 
various  platforms  available  in  the  battle  space,  with 
pages  for  the  data  from  each  sensor  on  web  portals 
for  each  friendly  platform  or  sensor  station. 

The  metrics  focus  on  the  content  of  the  messages 
and  what  they  contribute  to  command-center  deci¬ 
sions  as  opposed  to  the  speed  of  data  flow  through  the 
network.  These  improvements  can  be  demonstrated 
through  the  application  of  heterogeneous  sensor  data 
from  multiple  platforms  to  distributed  tracking  of 
unknown  contacts.  Modeling  and  simulation  of  in¬ 
formation  flow  in  the  battle  space  is  a  relatively  inex¬ 
pensive  way  to  depict  both  baseline  use  and  more 
efficient  future  uses  of  existing  sensors  and  their  data 
output,  without  costly  field  trials. 

Distributed  localization  and  tracking  can  be  dem¬ 
onstrated  by  cross  fixing  of  multiple  LOBs  obtained 
from  heterogeneous  (e.g.  acoustic,  magnetic)  sensor 
data.  Cross  LOBs  from  homogeneous  sensor  data  are 
used  in  ship  and  aircraft  navigation  to  determine  posi¬ 
tion.  However,  the  use  of  heterogeneous  sensor  data 
to  determine  the  position,  classification  and  identity 
of  unknown  contacts  and  potential  targets  in  the  battle 
space  has  not  been  utilized  effectively. 

In  the  KMDT  scenario,  sensor  data  are  transmitted 
in  messages  available  over  a  network.  Multiple  sensor 
ontologies  combined  in  a  single  format  can  increase 
understanding  of  message  content  and  provide  agents 
reference  material  for  selecting  the  right  platforms 
from  which  to  retrieve  data.  Intelligent  agents  can 
access  the  sensor  ontology,  obtain  tasking  from  com¬ 
mand  centers,  and  provide  alerts  when  critical 
thresholds  are  crossed.  The  agents  can  relieve  over¬ 
loaded  operators  by  retrieving  more  complete  infor¬ 
mation  from  existing  heterogeneous  sources.  The 
availability  in  the  battle  space  of  this  additional  in¬ 
formation  is  aimed  at  reducing  tracking  uncertainty 
and  targeting  errors. 


Data  from  sensors  include  LOB,  range,  spatial  ve¬ 
locity  from  Doppler  radar,  acceleration  from  several 
position  points,  pulse  repetition  rate,  peak,  frequency, 
etc.  Use  of  sensor  data  can  be  prioritizeds.  For  exam¬ 
ple,  the  order  of  usage  priority  could  be  as  follow.  1. 
Own  platform  -  passive  sensors,  2.  Own  platform  - 
active  sensors,  3.  Friendly  platform  -  passive  sensors, 
and  4.  Friendly  platform  -  active  sensors.  Similarly, 
data  selected  for  correlation  can  be  prioritized  as  fol¬ 
lows.  First  correlate  data  from  similar  sensor  types 
(e.g.  all  acoustic)  then  consider  data  from  dissimilar 
sensors  (e.g.  acoustic,  electro-optic)  and  sources. 

5.  Metrics  for  Ontology  Development 

Ontology  metrics  can  be  used  in  a  variety  of  inte¬ 
gration  applications.  For  example,  they  can  be  ap¬ 
plied  to  a  common  ontology  reference  prior  to  proc¬ 
essing  and  integration,  or  they  can  be  applied  to 
schema  matching  in  extensible  Markup  Language 
(XML)  integration  [9]. 

With  respect  to  general  statistics,  the  develop¬ 
ment  of  an  integrated  sensor  ontology  can  be  (racked 
with  simple  metrics.  One  metric  is  the  number  of 
initial  concepts  input  into  Protege/OWL.  Some  other 
metrics  associated  with  concept  acquisition  are  1)  the 
number  of  added  ontologies;  2)  the  total  number  of 
concepts  in  the  proposed  ontology  prior  to  inte¬ 
gration;  and  3)  the  number  remaining  in  the  inte¬ 
grated  ontology,  assuming  all  non-redundant  con¬ 
cepts  are  retained.  Metrics  associated  with  ontology 
integration  are  1)  the  number  of  redundant  concepts 
deleted  because  they  were  not  needed;  2)  the  number 
of  concepts  added  to  fill  gaps  that  became  apparent 
during  the  integration  process;  and  3)  the  number  of 
remaining  concepts  in  the  final  integrated  ontology. 
Still  another  dimension  of  metrics  is  to  count  the 
number  of  levels  in  the  ontology  hierarchy  and  the 
classes  or  instances  residing  at  each  level. 

In  addition  to  the  metrics  described  above,  a 
method  is  needed  to  characterize,  estimate,  and  even¬ 
tually  measure  disjunction  in  information  systems, 
and  particularly  in  ontology-integration  tasks.  Class 
cohesion  has  been  studied  in  object-oriented  systems 
and  metrics  have  been  developed  [1],  [2].  Ontologies 
are  hierarchical  structures  similar  to  structures  in 
object-oriented  systems.  The  cohesion  metrics  meas¬ 
ure  cohesion  between  members  of  the  same  class 
whereas  the  individual  disjunction  metric  described 
below  tracks  the  placement  of  the  same  concept  in 
related  ontologies. 

A  disjunction  metric  proposed  here  specifies  the 
degree  of  disjunction  in  ontologies  by  identifying  the 
level  of  generality  or  specificity  at  which  a  concept 


occurs  in  one  ontology,  compared  to  the  level  of  oc¬ 
currence  in  another  ontology.*  The  disjunction  metric 
is  useful  in  an  ontology-integration  application  when 
comparing  the  value  added  of  various  ontologies  that 
were  developed  separately  from  different  sources. 

To  apply  the  metric,  Dj  in  equation  (1)  below,  all 
levels  in  the  hierarchy  of  concepts  in  each  ontology 
must  be  labeled  with  1  representing  the  most  specific 
instances,  and  higher  numbers  representing  upper- 
level  ontologies. 

(1)  Dj  (Oi(Cj),  O^c*) ...  OpfcJ)  =  (i,  k, . . .  m) 

Equation  (1)  defines  the  disjunction  metric,  Dj  as  a 
set  of  levels  at  which  a  common  concept  occurs  in  a 
collection  of  ontologies.  In  (1),  “c”  is  a  concept  that 
occurs  at  level  “i”  in  ontology  1,  which  is  called 
“Oi.”  The  same  concept,  c,  occurs  in  ontology  2, 
called  “02,”  at  level  “k.”  Concept  “c”  also  occurs  at 
some  arbitrary  level  “m”  in  ontology  p,  called  “Op.” 
The  “...”  in  (1)  means  that  the  number  of  ontologies 
that  can  be  compared  in  this  manner  is  not  restriction. 
For  example,  equation  (2)  illustrates  the  disjunction 
metric  in  an  hypothetical  case  of  two  ontologies,  1 
and  2.  If  common  concept  “c”  found  at  level  3  in 
ontology  1,  were  also  found  at  level  5,  in  ontology  2, 
one  could  write  the  disjunction  metric  as  follows: 

(2)  Dj  (Oi(c3),  02(c5))  =  (3, 5) 

Equation  (1)  is  meant  to  express  disjunction  for  a 
single  concept.  However,  many  concepts  are  found  in 
any  meaningful  ontology.  To  measure  and  compare 
the  characteristics  of  various  ontologies,  an  overall 
disjunction  metric  is  needed  to  include  multiple  con¬ 
cepts,  not  just  one.  To  calculate  an  overall  estimate  of 
disjunction,  each  index  (i,  k,  ...  m)  can  be  averaged 
separately  across  a  group  of  concepts  that  occur  in  the 
same  ontology.  An  overall  disjunction  metric  for  two 
ontologies  can  be’  calculated  using  average  values  of 
the  levels  for  a  collection  of  “n”  concepts: 

(3)  < Dj  (Oj,  02)  >  =  (Xi/n,  Sk/n) 

where  the  instances  of  i  and  k  are  the  values  of  each 
pair  of  levels  found  for  each  common  concept.  To  use 
this  metric,  the  ontology  that  pertains  to  each  knowl¬ 
edge  base  (KB)  must  be  sufficiently  complete  to  lo¬ 
cate  the  corresponding  levels  in  the  ontologies.  An¬ 
other  way  to  conceptualize  the  disjunction  metric  in 
(2)  is  to  consider  that  a  concept  at  level  3  of  ontology, 
Oi,  is  equivalent  to  a  corresponding  concept  at  level  5 
of  ontology,  02.  The  usefulness  of  disjunction  metrics 
will  increase  when  a  more  standardized  way  to  organ¬ 
ize  an  ontology  is  developed.  Dj  and  <Dj>  will  de¬ 
pend  not  only  on  concepts  in  common  but  also  on  the 
structure  of  the  various  ontologies. 

For  example,  consider  the  calculation  of  an  overall 
disjunction  metric  so  that  “n”  in  (3)  is  3: 


(4)  Dj  (0,(c3),  02(c5))  =  (3,5) 

(5)  DKOKc^OjCcd)  =  (2,4) 

(6)  Dj  (OjCci),  02(c3))  =  (1,  3) 

(7)  <  Dj  (Oj,  02)>  =  ( (3+2+l)/3,  (5+4+3)73  ) 

(8)  <  Dj  (Oj,  02)  >  =  (2,4) 

An  assumption  in  the  above  equations  is  that  the 
each  equation  addresses  a  distinct  concept.  If  the 
overall  disjunction  metrics  are  low  (e.g.  (1,2))  it  indi¬ 
cates  that  the  ontologies  have  common  concepts  at  the 
same  level  of  granularity,  which  is  important  to  know 
for  integration  purposes.  It  is  also  a  signal  to  look  for 
duplicate  concepts  and  delete  any  redundant  informa¬ 
tion.  If  the  metrics  are  high,  (e.g.  5,7)  it  implies  that: 
1)  The  ontologies  proposed  for  integration  may  con¬ 
tain  concepts  that  have  been  overlooked  and  therefore 
are  missing  in  the  integrated  ontology,  or  2)  The  pro¬ 
posed  addition  to  the  ontology  does  not  relate  to  the 
main  topic.  Dj  will  depend  on  the  structure  of  the 
various  ontologies  and  therefore  can  provide  at  a 
glance  some  insight  regarding  the  relative  ontology 
hierarchies.  Low  numbers  for  very  general  concepts, 
such  as  1  or  2,  indicate  a  very  flat  ontology  whereas 
higher  numbers,  such  as  the  ones  in  (4),  indicate 
more  levels  of  specialization/generalization. 

6.  Intelligent-Agent  Performance  Metrics 

Metrics  and  statistics  can  be  used  to  evaluate  and 
document  the  behavior  of  agents  in  simulations.  The 
following  metrics  can  track  the  activity  of  the  agents: 

1.  Number  of  web  portals  accessed  to  search  for  the 
desired  data, 

2.  Number  of  relevant  data  retrieved  from  each  site, 

3.  Number  of  successful  data  retrievals  vs.  the  number 
of  agent  deployments  on  an  individual-agent  basis, 

4.  Same  statistics  as  in  item  3  above,  but  summarized 
to  include  all  agents  deployed  during  a  given  time 
period  in  the  simulation. 

The  following  metrics  can  help  analyze  agent  errors: 

1.  Number  of  irrelevant  data  retrieved  from  each  site, 

2.  Number  of  incorrect  data  retrieved  from  each  site, 

3.  Number  of  correct  data  that  the  agents  could  have 
retrieved  but  did  not  (a  manual  analysis  that  involves 
keeping  track  of  all  possible  alternatives  in  the  simu¬ 
lation  and  comparing  the  results  to  the  alternatives.) 

The  degree  of  uncertainty  in  the  location  of  an  un¬ 
known  platform  or  target  depends  on  many  factors, 
including  the  correctness  of  the  retrieved  sensor  data 
and  the  angle  between  the  lines  of  bearing.  Thus, 
agent  performance  can  be  monitored  by  calculating 
the  angle  between  the  lines  of  bearing  for  each  suc¬ 
cessful  data  retrieval  that  results  in  a  correctly  identi¬ 


fied  cross  line  of  bearing.  Equation  10  gives  a  for¬ 
mula  for  E,  the  error  that  results  from  the  departure 
from  a  right  angle  between  two  lines  of  bearing  situ¬ 
ated  at  angle,  (j>.  This. metric  can  be  used  to  compare 
data  retrievals  among  the  various  agents  so  that  the 
line  of  bearing  with  the  lowest  value  of  E  can  be  se¬ 
lected  in  cases  where  the  agents  retrieve  multiple 
cross  lines  of  bearing. 

(10)  E  =  1  -  ( sin  (<j>)  | 

One  way  to  collect  the  data  on  agent  errors  is  to 
save  the  history  of  the  simulation  scenario  in  a  log 
file  for  later  analysis.  Using  these  metrics,  the  agent 
algorithm  for  platform  selection  can  be  tested  as  well 
as  the  correctness  of  the  data  retrieved. 
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